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INTRODUCTION 

For many yean, email groups of computers have been 
interconnected In various ways. Only recently, how¬ 
ever; has the interaction of computers and communica¬ 
tions become an important topic in its own right.** In 
1968, after considerable preliminary investigation And 
discussion, the Advanced Research Projects Agency 
of the Department of Defense (ARPA) embarked on 
the implementation of a new kind of nationwide 
computer interconnection known as the ARPA Net¬ 
work. This network will initially interconnect many 
dissimilar computers at ten ARPA-aupported research 
centers with 60-kilobit common-carrier circuits The 
network may be extended to include many other 
locations and circuits of higher bandwidth. 

The primary goal of the ARPA project is to permit 
persons and programs at one research center to access 
data and use interactivciy programs that exist and 
run in other computers of the network. This goal may 
represent a major step down the path taken by com¬ 
puter time-sharing, in the sense that the computer 
resources of the various research centers arc thus 
pooled and directly accessible to the entire community 
of network participants. 

Study of the technology and tariffs of available 
communications facilities showed that use of con¬ 
ventional line switching facilities would be economically 
and technically inefficient. The traditional method of 
routing information through the common-earner 
switched network establishes a dedicated path for each 
conversation. With present technology, the time 
required for this task is on the order of seconds. For 
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voice communication, that overhead time is negligible, 
but in the case of many short transmissions, such as 
may occur between computers, that time is excessive. 
Therefore, ARPA decided to build a new kind of 
digitai communication system empioying wideband 
leased lines and message switching, wherein a path is 
not established in advance and each message carries an 
address. In this domain the project portends a possible 
major change in the character of data communica¬ 
tion services in the United States. V 

In a nationwide computer network, economic cqa-- 
^derations also mitigate against a wideband leased 
line configuration that is topologically fully connected. 
In a non-fully connected network, messages • must 
normally traverse several network nodes in going from 
source to destination. The ARPA Network is designed 
on this principie and, at each node, a copy of the mes¬ 
sage is stored until it is 6afely received at the following 
node. The network is thus a store and forward system 
and as such mU6t deal with problems of routing, buffer¬ 
ing, synchronization, error control, reliability, and 
other related issues. To insulate the computer centers 
from these problems, and to insulate the network from 
the problems of the computer centers, ARPA decided 
to place identical small processors at each network 
node, to interconnect these smali processors with 
leased common-carrier circuits to form a subnet, and 
to connect each research computer center into the net 
via the local email processor. In this arrangement the 
research computer centers are called Haste and the 
small processors are called Interface Message Processors, 
or IMPs. (See Figure 1.) This approach divides the 
genesis of the ARPA Network into two porta: (I) 
design and implementation of the IMP subnet, and 
(2) design and implementation of protocols and tech¬ 
niques for' the sensible utilisation of the network by 
the Hosts. 

Implementation of the subnet Involves two major 
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Figure 6—The IMP 


nance, debugging, and system modification; in normal 
operation, the IMP runs without any moving parts 
except fans. Within the cabinet, ep'ace has been re* 
served for an additional 4K memory. Figure 6 Is & 
picture of an IMP, and Figure 7 shows its configura¬ 
tion. 

Ruggedisation of computer hardware for use in 
friendly environments is somewhat unusual; however, 
we felt that the considerable difficulty that IMP 
failures can cause the network justified this step. 
Although the ruggediied unit is not fully “qualified" 
to MIL specs, it does have greater resistance to tem¬ 
perature variance, mechanical shock and vibration, 
radio frequency interference, and power line noise. 
We are confident that this ruggediaation will increase 
the mean time to failure. 

Modular Host and modem Interfaces allow an IMP 
to be individually configured for each network node. 
The modularity, however, does not take the form of 
pluggable unite and, except for the possibility of 
adding interfaces into reserved frame space, recon¬ 



figuration is impractical. Various configurations allow 
for up to two Hosts and five modems, threo Hosts and 
four modems, etc. Each modem interface requires 
approximately one-fourth the amount of logic used in 
the C.P.XJ. Tho Host interface is somewhat smaller 
(about one-sixth of the C.P.U.). 

Interfaces to the Host and to the modems have 
certain common characteristics. Both are full duplex, 
both may be crosspatched under program control to 
test their operation, and both function in the a&me 
general manner. To send a pocket, the IMP program 
eets up memory pointers to the pocket and then 
activates the interface via a programmable control 
pul6e. The Interface takes successive words from the 
memory using ite assigned output data channel and 
transmits them bit-scrially (to the Host or to the 
modem). When the memory buffer has thus been 
emptied, the interface notifies the program via an 
interrupt that the job has been completed. To receive 
information, the program first sets pointers to the 
allocated space in the memory into which the Informa¬ 
tion is to flow. Using a control pulse It then readies 
the interface to receive. When information starts to 
arrive (here again bit-serialiy), It is assembled into 
Id-bit words and stored into the IMP memory. When 
either the allocated memory space is full or the end of 
the data train Is detected, the interface notifies the 
program via on interrupt. 

The modem interfaces deal with the phone lines in 
terms of 8-bit characters; the interfaces idle by sending 
and receiving a sync pattern that keeps them in charac¬ 
ter sync. Bit sync is maintained by the modems them¬ 
selves, which provide both transmit and receive dock¬ 
ing signals to the interfaces. When the program initiates 
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TABLE I—Program Data Structures 

5000 WORDS—MESSAGE BUFFER STORAGE 
120 WORDS—QUEUE POINTERS 
300 WORDS—TRACE BLOCKS 
100 WORDS—REASSEMBLY BLOCKS 
150 WORDS—ROUTINO TABLES 
400 WORDS—LINK TABLES 
800 WORDS—3TATISTIGS TABLES 


contains the remaining five programs and deals with 
initialization, debugging, testing, statistics gathering 
and tracing. After a brief description of data struc¬ 
tures, we will discuss packet processing in some detail. 

Buffer allocation, queue*, and table* 

The major system data structures (6ee Table I) 
consist of buffers and tables. The buffer storage space 
is partitioned into about 70 fixed length buffers, each 
of which ia used for storing a single packet. An unused 
buffer is chained onto a free buffer list and is removed 
from this list when it is needed to store an incoming 
packet. A packet, once stored in a buffer, is never 
moved. After a packet has been successfully passed 
along to its Host or to another IMP, its buffer is re¬ 
turned to the free list. Tho buffer epace is partitioned 
in euch a way that each process (store and forward, 
traffic, Host traffic, tie.) is always guaranteed some 
buffers. For the sake of program speed and simplicity, 
no attempt Is made to retrieve the space wasted by 
partially filled buffers. 

In handling store and forward traffic, all processing 
is on a per packet basis. Further, although traffic to 
and from Hosts is composed of the IMP 

rapidly converts to dealing with pockets; the Host 
transmit* a message as a single unit but the IMP 
takes it one buffer at a time. As each buffer is filled, 
the program select* another buffer for input until the 
entire message has been provided for. These successive 
buffers will, In general, be scattered throughout the 
memory. An equivalent inverse process occurs on 
output to the Host after all packets of the message 
have arrived at the destination IMP. No attempt is 
ever made to collect the packet* of a message into a 
contiguous portion of the memory. 

Buffers currently in use are either dedicated to an 
incoming or an outgoing packet, chained on a queue 
a-vvaiting processing by the program, or being processed. 
Occasionally, a huffer may be simultaneously found on 
two queues; this situation can occur when a packet is 
waiting on ono queue to be forwarded and on another 
to be acknowledged. 


There arc four principal types of queues: 

Task: Packets received on Host channels are placed 
on the Host task queue. All received acknowledg¬ 
ments, dead Host and routing information, / heard 
you and hello packet* are placed on the system task 
queue; all other packet* from the modems are placed 
on the modem task queue. The program services the 
system task queue first, then the Host task queue, and 
finally the modem task queue. 

Output: A separate output queue la constructed for 
each modem channel and each ' Host channel. Each 
modem output queue is subdivided into an acknowl¬ 
edgment queue, a priority queqe, a RFNM queue, 
and a regular message queue, which are serviced in 
that order. Each Host output queue it subdivided Into 
a control message queue, a priority queue, and a 
regular message queue, which are also serviced in the 
indicated order. 

jSsnl; A separate queue for each modem channel con¬ 
tains packets that have already been transmitted on 
that line but for which no acknowledgment has yet 
been received. 

Reassembly: The reassembly queue contains those 
packets that are being reassembled into messages for 
the Host. 

Tabiea in core are allocated for the Btoragc of queue 
pointers, for trace blocks, for reassembly information, 
for statistics, and for linkB. Most noteworthy of these 
is the link table, which is used at the source IMP for 
assignment of message numbers and for blocking and 
unblocking links, and at the destination IMP to 
verify message numbers for sequence control. 

Packet Bow and program structure 

Figure 9 is a schematic drawing of packet process¬ 
ing; the processing programs are described below. 

The Host-to-IMP routine (H I) handles messages 
being transmitted from the local site. The routine 
uses the leader to construct a header that is prefixed 
to each packet of the message. It also creates a link 
for the message If necessary, blooka the link, puts the 
packets of the message on the Host task queue for 
further processing by the task routine, and triggers 
the programmable task interrupt. The routine then 
acquires a free buffer and seta up a new input. The 
routine teste a hardware trouble indicator, verifies the 
message format, and chocks whether or not the destina¬ 
tion is dead, the link table Is full, or the link blocked. 
The routine is serially reentrant and services all Hosts 
connected to the IMP. 
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The Modem-lo-lMP routine (M —♦ I) handies inpute 
from the modems. This routine consists of several 
identical routines, one for ec.ch modem channel. (Such 
duplication is useful to obtain higher speed.) This 
routine seta up an input buffer (normally obtained 
from the free list), places the received packet on the 
appropriate task queue, and triggers the programmable 
task Interrupt. Should no free buffers be available for 
input, the buffer at the head of the modem task queue 
is preempted. If the modem task queue is also empty, 
the received packet is discarded by setting up its 
buffer for input. However, a sufficient number of free 
buffers are specifically reserved to assure that received 
acknowledgments, routing packets, and tho like are 
rarely discarded. 

The Uuk routine uses the header information to 
direct packets to their proper destination. The task 
routine is driven by the task interrupt, which is set 
whenever a pseket is put on a task queue. The task 
routine routes packets from the Host task queue onto 
an output queue determined from the routing algorithm. 

For each packet on the modem task queue, the task 
routine first determines whether sufficient buffer space 
is available. If the IMP Has a shortage of store and 
forward buffers, the buffers on the modem task queue 
are simply returned to the free list without further 
processing. Normally, however, an acknowledgment 
packet is constructed and put near the front of the 
appropriate modem output queue. The destination of 
the packet is then inspected. If the packet is not for 
the local site, the routing algorithm selects a modem 
output queue for tho packet. If o packet for the local 
site is a RFNM, the corresponding link is unblocked 
and the RFNM is put on a queue to the Host. If the 
packet is not a RFNM, it is joined with others of the 



same message on the reassembly queue. Whenever a 
message is completely reassembled, the packets of 
the message are put on an output queue to the Host 
for processing by the IMP-to-Host routine. 

In processing the system task queue, the task routine 
returns to the free list those buffers from the sent 
queue that have been referenced by acknowledgments. 
Any packets skipped over by an acknowledgment are 
designated for retransmission. Routing, / heard you, 
and hello packets are processed In a straightforward 
fashion. 

The IMP-to-Modevx routine (I —* M) transmits 
successive packets from the Modem output queue. 
After completing the output, this routine places any 
packet requiring acknowledgment on the sent queue. 

The IMP-lo-Host routine (7 —♦ H) sets up successive 
outputs of packets on the Host output queues and 
constructs a RFNM for each non-control message 
delivered to a Host. RFNM packets are returned to 
the system via the Host task queue. 

The time-out routine is started every 25.0 msec 
(called the time-out period) by a clock interrupt. 
The routine has three sections: the fast* time-out 
routine, which "wakes up" any Host or modem inter¬ 
rupt routine that has languished (for example, when 
the Host input routine could not immediately Btart a 
new input because of a shortage in buffer Bpace); the 
middle time-out routine, which retransmits any packets 
that have been too long on a modem sent queue; and 
the slow* time-out routine, which marks lines as alive 
or dead, updates the routing tables and does long 
term garbage collection of queues and other data 
structures. (For example, it protcote the system from 
the cumulative effect of such failures as a lost packet 
of a multiple pocket message, where buffers are tied 
up in message reassembly.) It also deletes links auto¬ 
matically after 15 seconds of disuse, after 20 minutes 
of blocking, or when an IMP goes down. 

These three routines arc executed in the following 
pattern: 

FFFF FFFF FFFF FFFF FFFF FFFF ... 

M M M M M 

8 

and, although they run off a common interrupt, are 
constructed to allow faster routines to interrupt slower 
ones should a slower routine not complete execution 
before the next time-out period. 

The JmJk routine enters, examines, and deletes entries 
from the link table. A table containing a separate 
message number entry for many links to every pouJblo 
Host would be prohibitively large. Therefore, the 
table contains entries only for each of 03 total out- 


Figure 9—Internal pseket flow 





t 


KATIE HAFNER 


1 512 *T6 1966 


562 Spring Joint Computer Conference, 1070 


going link* it any Host site. Hashing is used to epeed 
accessing of this tabic, but the Link program is still 
quite costly; it uses about ten percent of both speed 
and space in a conceptually trivial task. 


Initialisation and background loop 

The IMP program starts in an initialization section 
that builds the initial data structures, prepares for 
inputs from modem and Host channels, and resets ali 
program switches to their nominal state. The program 
then falls into the background loop, which is an end* 
leasly repeated scries of low-priority subroutines that 
are interrupted to handle normal traffic. 

The programs in the IMP background loop perform 
a variety of functions: TTY is used to handle the IMP 
Teletype traffio; DEBUG, to inspect or change IMP 
core memory; TRACE, to transmit collected informa¬ 
tion about traced packets; STATISTICS, to take and 
transmit network and IMP statistics; PARAMETER- 
CHANGE, to alter the values of selected IMP pa¬ 
rameters; and DISCARD, to throw away packets. 
Selected Hosts and IMPs, particularly the Network 
Measurement Center and the Network Control Center, 
will find it necessary or useful to communicate with 
one or more of these background loop programs. So 
that these programs may send and receive messages 
from the network, they are treated as "fake Hosts". 
Rather than duplicating portions of the large IMP-to- 
Host and Host-to-IMP routines, the background loop 
programs arc treated as if they were Hosts, and they 
con thereby utilize existing programs. The "For IMP" 
bit or the "From IMP" bit in the leader indicates 
that a given message is for or from a fake Host program 
in the IMP. Almost all of the background loop is 
devoted to running these programs. 

The TTY program assembles characters from the 
Teletype into network messages and decodes network 
messages Into characters for the Teletype; TTY's 
normal message destination is the DEBUG program 
at its own IMP; however, TTY can be made to com¬ 
municate with any other IMP Teletype, any other 
IMP DEBUG program or any Host program with 
compatible format. 

The DEBUG program permits the operational 
program to be inspected and changed. Although ite 
normal message source is tho TTY program at ite 
own IMP, DEBUG will respond to a message of the 
correct format from any source. This program is 
normally inhibited from changing the operational 
IMP program; local operator intervention is required to 
activate the program's full power. 

The STATISTICS program collects measurements 


about network operation and periodically transmit! 
them to the Network Measurement Center. This 
program sends but does not receive messages. STA¬ 
TISTICS has a mechanism for collecting measure¬ 
ments over 10-second intervals and for taking half- 
second snapshots of IMP qusue iengths and routing 
tables. It can also generate artificial traffic to load 
the network. When turned on, STATISTICS uses 10 
to 20 percent of the machine capacity and generates a 
noticeable amount of phone iine traftio. 

Other programs in the background loop drive local 
status lights and operate the parameter change routine. 
A thirty-two word parameter table controls the opera¬ 
tion of the TRACE and STATISTICS programs and 
includes spares for expansion; the PARAMETER- 
CHANGE program accepts messages that change 
these parameters. 


Control organization 

It is characteristic of the IMP system that many 
of the main programs are entered both as subroutine 
calls from other programs and as interrupt calls from 
the hardware. The resulting control structure is shown 
in Figure 10. The programs arc arranged in a priority 
order; control passes upward in the chain whenever a 
hardware interrupt occurs or the current program 
decides that the time has come to run a higher priority 
program, and control passos downward only when 
the higher priority programs aro finished. No program 
may execute cither itself or a lower priority program; 
however, a program may freely execute a higher pri¬ 
ority program. This rule is similar to the usual rules 
concerning priority interrupt routines. 

In one important case, however, control must pass 
from a higher priority program to a lower priority 
program—namely, from the several input routines to 
the TASK routine. For this special cose, the com¬ 
puter hardware was modified to include a low-priority 
hardware Interrupt that can be sat by (he proa ram. 
When this interrupt has been honored (i.e., when all 
other interrupts have been serviced), the TASK 
routine is executed. Thus, control is directed where 
needed without violating the priority rules. 

Some routines must occasionally wait for long inter¬ 
vale of time, for example, when the Host-to-IMP 
routine must wait for a link to unblock. Stopping the 
whole system would be intolerable; therefore, should 
the need arise, such a routine is dismissed, and the 
TIMEOUT routine will later transfer control to the 
waiting routine. 

The control structure and the partition of responsi¬ 
bility among various programs achieve the following 
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timing goals; 

1. No program stops or delays the system while 
waiting for an event. 

2. The program gracefully adjusts to the situation 
where the machine becomes compute-bound. 

3. The Modem-to-IMP routine can deliver its 
eurrent packet to the TASK routine before the 
neat packet arrives and can always prepare for 
successive packet inputs on each line. This 
timing is critical because a slight delay here 
might require retransmission of the entire pocket. 
To achieve this result, separate routines (one per 
phone line) interrupt each other freely after new 
buffers have been set up. 

4. The program will almost always deliver packets 
waiting to be sent as fast aa they can be accepted 
by the phone line. 

5. Neceesary periodic processes (in the time-out 
routine) are always permitted to run, and do 
not interfere with input-output processes. 

Support aoftuwe 

Designing a real-time program for a small computer 
with many high rate I/O channels is a specialised kind 
of software problem. The operational program requires 
not only unusual techniques but also extra software 
tools; often the importance of such extra tools is not 
recognized. Further, even when these issues are recog¬ 
nised, the effort needed to construct such tools may be 
seriously underestimated. The development of the 
IMP system required the following kinds of supporting 
software: 

1. Programs to test the hardware. 

2. Tools to help debug the system. 

3. A Host simulator. 

4. An efficient assembly process. 

So far, three hardware test programs have been 
developed, The first and largest is a complete program 
for tea ting all the special hardware features in the 
IMP. This program permits running any or all of the 
modem interfaces In a crosspatched mode; it even 
permits operating together tevcrol IMPs in a test 
mode. The second hardware teet program runs a 
detailed phone line test that provides statistics on 
phone line errors. The final program simulates the 
modem interface check register whose complex be¬ 
havior is otherwise difficult,to predict. 

The software debugging tools exist In two forms. 
Initially we designed a ample stand-alone debugging 
program with the capability to do little more than 
examine and change individual core registers from the 



oonsole Teletype. Subsequently, we embedded a 
version of the stand-alone debugging program into 
the operational program. This operational debugging 
program not only provides debugging assistance at a 
single location but also may be used in network tutinQ 
and network debugging. 

The initial implementation of the IMP software 
took place without connecting to a true Host. To 
permit checkout of the Host-related portions of the 
operational program, we built a "Host Simulator" 
that takes input from the console Teletype and feeds 
the Host routines exactly as though tbe input had 
originated in a real Host. Similarly, output messages 
for a destination Host are received by the simulator 
and typed out on the console Teletype. 

Without recourse to expensive additional periph¬ 
erals, the assembly facilities on the DDP-516 are 
inadequate for a large program. (For example, a listing 
of the IMP program would require approximately 20 
hours of Teletype output.) We therefore used other 
locally available facilities to aeeist In the assembly 
process. Specifically, we used a FDP-1 text editor to 
oompose and edit the programs, assembled on the 
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TABLE II—Transit Timas And Ma»|i Rates 



Minimum 

Maximum 

SINGLE WORD UBSSAOS 

Transit Tims 

5 msec 

50 msec 

Round-trip 

!0 msec 

{00 msec 

Mar. Message Rite/Link 

100/MA 

10/mo 

SINGLE FULL PACKET MESSAGE 


Transit Time 

45 msec 

140 mate 

Round-trip 

50 msec 

190 msec 

Max. Massage Rate/Link 

20/..C 

5/see 

8-PACKET MESSAGE 

Transit Time 

265 msec 

360 mate 

Round-trip 

105 msec 

320 mace 

Max. Manage Rate/Link 

5/aec 

3/sec 


DDP-516, and Hated the program on the SD5 940 
line printer. Use of this assembly process required 
minor modification of existing PDP-1 and SDS 040 
Bupport software 


PROJECTED IMP PERFORMANCE 

At this writing, the subnet has not yet been sub¬ 
jected to realistic load conditions; consequently, very 
little experimental data is available. However, we have 
made some estimates of projected performance of the 
IMP program and we describe these estimates below. 


Host traffic and message delays 

In the subnet, the Hoet-to-Host transit time and 
the round-trip time (for RFNM receipt) depend upon 
routing and message length. Since only one message 
at a time may be present on a given link, the reciprocal 
of the round-trip delay Is the maximum message rate 
on a link. The primary factors affecting subnet delays 
are: 

• Propagation delay: Electrical propagation time 
In the Bell system is estimated to be about 10 
pace per mile. Cross country propagation delay is 
therefore about 30 msec. 

• Modem transmission delay: Because bits enter 
and leave an IMP at a predetermined modem hit 
rate, a packet requires a modem transmission 
time proportional to its length (20 psec per bit on 
a 50-kilobit line). 


• Queueing delay: Time spent waiting in the IMP 
for transmission of previous packete on a queue. 
Such waiting may occur either at an intermediate 
IMP or in connection with terminal IMP trans¬ 
missions into the destination Host. 

• IMP processing delay: The time required for the 
IMP program to process a packet is about 0.35 
msec for a store-and-forward packet. 

Because the queueing delay depends heavily upon 
the detailed traffic load in the network, an estimate of 
queueing delay will not be available until we gain 
considerable experience with network operation. In 
Table II, we show an estimate of tho one-way and 
round-trip transit times and the corresponding maxi¬ 
mum message rate per link, assuming the negligible 
queueing delay of a lightly loaded net. In this table, 
“minimum” delay represents a short hop between 
two nearby IMPs, and “maximum” delay represents a 
cross-country path involving five IMPs. In all 
the delays are well within the desired half-second 
goal. 

In a lightly-loaded network with a mixture of nearby 
and distant destinations, an example of heavy Host 
traffic into its IMP might be that of 20 Jinks carrying 
ten single-word messages per second and four more 
links, each carrying one eight-packet message per 
eeoond. 


Computational load 

In general, a line fully loaded with short packets 
will require more computation than a line with all 
long packete; therefore the IMP can handle more 
lines In the latter esse. In Figure 11, we show a curve 
of the computational utilisation of the IMP as a func¬ 
tion of message length for fully-loaded communication 
lines. For example, a 50-kllobit line fully loaded in both 
directions with one-word messages requires slightly 
over 13 percent of the available IMP time. Since a 
line will typically carry a variety of different length 
packete, and each line will be leas than fully loaded, 
the computational load per line will actually be much 
lees. 

Throughput is defined to be the maximum number 
of Host data bits that may traverse an IMP each 
second. The actual number of bite entering the IMP* 
per second is somewhat larger than the throughput 
because of such overhead as headers, RFNMs, and 
acknowledgments. The number of bits on the lines are 
stiil larger because of additional line overhead Bueh 
os framing and error control characters. (Each packet 
on the phone line contains seventeen characters of 
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overhead, nine of which are removed before the packet 
enters an IMP,) 

The computational limit on the IMP throughput is 
approximately 700,000 bits per second. Figure 12 
shows maximum throughput as a function of message 
length. The difference between the throughput curve 
and the line traffic curve represents overhead. 


DISCUSSION 

In this section we state some of our conclusions about 
the design and implementation of the ARPA Network 
and comment on possible future directions. 

We are convinced that use of an IMP-Iike device is 
a more sensible way to design networks than is use of 
direct Host-to-Hoet connection. First, for the subnet 
to serve a store-and-forward role, its functions must be 
independent of Host computers, which may often be 
down for extended periods. Second, the IMP program 
is very complex and is highly tailored to the I/O struc¬ 
ture of the DDF-5I6; building such complex functions 
into special I/O units of each computer that might 
need network connection is probably economically 
inadvisable. Third, because of the desirability of 
having several Host computers at a given site connect 
to the network, it is both more convenient and more 
economic to employ IMPs than to provide all the 
network functions in each of the Host computers. The 
whole notion of a network node serving a multiplexing 
function for complexes of local Hosts and terminals 
lends further support to this conclusion. Finally, 
because we were led to a design having tome inter- 
IMP dependence; we found it advantageous to have 
identical units at each node, rsther than computers 
of different manufacture. 

Considering the multiplexing issue directly, it now 
seems clear that individual network nodes will be 
connected to a wide variety of computer and terminal 
complexes. Even the initial ten-node ARPA Network 
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Figure 12—IMP throughput 


includes one Host organisation that has chosen to 
submultiplex several computers via a single Host 
connection to the IMP. We are now studying variants 
of the IMP design that address this multiplexing 
issue, and we also expect to cooperate with other 
groups (suoh as at the National Physical Laboratory 
In England) that are studying such multiplexing 
techniques. 

The increasing interest in computer networks will 
bring with it an expanding interaction between com¬ 
puters and communication circuits. From the outset, 
we viewed the AKPA Network as a systems engineer¬ 
ing problem, including the portion of the system sup¬ 
plied by the common carriers. Although we found the 
carriers to be properly concerned about circuit per¬ 
formance (the basic circuit performance to date has 
been quite satisfactory')! w ® found it difficult to work 
with the carriers cooperatively on the technical details, 
packaging, and implementation of the communication 
circuit terminal equipment; as a result, the present 
physical installations of circuit terminal equipment 
ore at best inelegant and inconvenient. In the longer 
run, for reasons of economy, performance, and reliabil¬ 
ity, circuit terminal equipment probably should be 
integrated more closely with computer input/output 
equipment. If the carriers are unable to participate 
conveniently in such integrations, we would expect 
further growth of a competing circuit terminal equip¬ 
ment industry, and more prevalent common carrier 
provision of bare circuits. 

Another aspect of network growth and development 
is the requirement to connect different rate com¬ 
munication eircuite to IMP-like devices as .a function 
of the particular application. In our own IMP design, 
although there are limitations on total throughput, 
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the IMP can be connected to carrier circuit* of any 
bit rate up to about 250 kilobits; similarly, the inter¬ 
face to a Host computer can operate over a vide 
range of bit rates. We feel that this flexibility is very 
important because the economies of carrier offering®, 
m well aa the user requirements, are subject to sur¬ 
prisingly rapid change; even within the time period 
of the present implementation, we have experienced 
such changes. 

At this point, we would like to discuss certain aspeots 
of the implementation effort This project required 
the design, development, and installation of a very 
complex device in a rather short time scale. The diffi¬ 
culty in producing a complex system is highly de¬ 
pendent upon the number of people who ore simul¬ 
taneously involved. Small groups can achieve complex 
optimizations of timing, storage, and hardware/ 
software interaction, whereas iarger groups can seldom 
achieve such optimizations on a reasonable time 
scale. We those to operate with a very small group of 
highly talented people. For example, all software, 
including software tools for assembly, editing, debug¬ 
ging, and equipment testing as well as the main opera¬ 
tional program, involved effort by no more than four 
peopio at any time. Since so many computer system 
projeets Involve much larger groups, we feel it is worth 
calling attention to this approach. 

Turning to the future, we plan to work with the 
ARPA Network project along several technical direc¬ 
tions: (1) the experimental operation of the network 
and any modifications required to tune its perfor¬ 
mance; (2) experimental operation of the network with 
higher bandwidth circuits, e.g., 230.4 kilobits; (3) a 
review of IMP variants that might perform multi¬ 
plexing functions; (4) consideration of techniques for 
designing more economical and/or more powerful 
IMPs; and (6) participation with the Host organiza¬ 
tions in the very sizeable problem of developing tech¬ 
niques and protocols for the effective utilization of 
the network. 

On a more global level, we anticipate an explosive 
growth of message switched computer networks, not 
just for the interactive pooling of resource*, but for 
the ample conveniences and economies to be obtained 
for many classes of digital data communication. We 
believe that the capabilities inherent in the design of 
even the present subnet have broad application to 
other data communication problems of government 
and private industry. 
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